The mean normal body temperature was held to be 37$^{\circ}$C or 98.6$^{\circ}$F for more than 120 years since it was first conceptualized and reported by Carl Wunderlich in a famous 1868 book. In 1992, this value was revised to 36.8$^{\circ}$C or 98.2$^{\circ}$F.
In this exercise, you will analyze a dataset of human body temperatures and employ the concepts of hypothesis testing, confidence intervals, and statistical significance.
Answer the following questions in this notebook below and submit to your Github account.
You can include written notes in notebook cells using Markdown:
In [1]:
%matplotlib inline
from matplotlib import pyplot as plot
import pandas as pd
from scipy import stats
from statsmodels.api import qqplot
import numpy as np
In [2]:
df = pd.read_csv('data/human_body_temperature.csv')
In [3]:
# adding temp in degrees celsius
df['tempC'] = df.temperature.apply(lambda x: (x-32)*(5/9.))
In [4]:
df.describe().apply(lambda x: np.round(x,2))
Out[4]:
In [5]:
df.groupby("gender").describe().apply(lambda x: np.round(x,2))
Out[5]:
Notably the mean temperature rounds to roughly 37 °C, which then converts to 98.6, suggesting the 98.6 value may be a result of rounding followed by conversion, rather than conversion followed by rounding.
In [6]:
# ignore error message. without the .show() qqplot has some buggy behavior when run with %matplotlib inline
print "normal q-q plot for the sample"
qqplot(df.temperature,line='45',fit=True).show()
The majority of the data appears to fit the normal distribution relatively well. However, the lowest and highest quantiles show less of a good fit. There's also some apparent discreetness in the data, which apparently arises from rounding in the original data (see documentation). Splitting by gender, the women show less of a good fit to the normal distribution than men.
In [7]:
print "normal q-q plot for women in the sample"
qqplot(df[df.gender=="F"].temperature,line='45',fit=True).show()
In [8]:
print "normal q-q plot for men in the sample"
qqplot(df[df.gender=="M"].temperature,line='45',fit=True).show()
Upper histogram is the distribution for women, lower for men.
In [9]:
df.groupby("gender").hist(column=["temperature"],layout=(2,1),sharex=True)
Out[9]:
These aren't precisely normal looking, women moreso than men, but they aren't particularly badly off. However, given that the normality assumption is maybe not totally justified here, a t test might be more appropriate than a z test, being more robust with regard to this assumption.
In [10]:
genderBin = np.where(df.gender=="F",1,0)
genderBin
plot.scatter(df.temperature,df.heart_rate,c=genderBin,cmap='winter')
Out[10]:
In [11]:
print "Correlation between heart rate and temperature is %.3f (p = %.3f)" %stats.pearsonr(df.temperature,df.heart_rate)
So the relationship is significant and is large enough that it isn't necessarily something to be ignored. While some of this might be attributed to the gender differences in heart rate, there's a still a lot of overlap. Will address this is a bit more detail below.
In [12]:
ttest1 = stats.ttest_1samp(df.temperature,98.6)
print "The t-statistic is %.3f and the p-value is %s" % ttest1
So, the mean here is significantly different from 98.6. This is not particularly shocking, given the mean of 98.25 calculated above. Constructing a 95% confidence interval around the mean, we find that it doesn't contain 98.6
In [13]:
n = len(df.temperature)
m = df.temperature.mean()
se = stats.sem(df.temperature)
h = se * stats.t.ppf(0.975, n-1)
print "Lower: %.2f, Upper: %.2f" %(m-h,m+h)
For comparison, a z-test gives quite similar answers:
In [14]:
from statsmodels.stats.weightstats import ztest
ztest(x1=df.temperature,value=98.6)
Out[14]:
In [15]:
np.array(df.temperature)
Out[15]:
This is fairly difficult to say. Assuming nobody in the sample was seriously ill when the measurements were taken, there's a non-trivial range of variability (sample values range between 96 and 100 °F). Given that the distribution is roughly normal, 2 sds from the mean might be a useful guideline. This would give the range 96.79 to 99.71 °F, corresponding fairly well to Wikipedia's listed values for the observed range.
Do women and men show a difference in mean body temperature?
In [16]:
fTemp = df[df["gender"] == "F"].temperature
mTemp = df[df["gender"] == "M"].temperature
ttest2 = stats.ttest_ind(fTemp,mTemp)
print "The difference in means is %.3f degrees" % (fTemp.mean() - mTemp.mean())
print "The t-statistic is %.3f and the p-value is %.3f" % ttest2
So, the (small) differences in the estimated means calculated above also appear to be significant, although for the data shows a lot of overlap (see summary statistics above).
In [17]:
import statsmodels.formula.api as smf
In [18]:
lm = smf.ols(formula='temperature ~ heart_rate + gender', data=df).fit()
print lm.summary()
Having heart rate and gender both in the model does not result in either individual effect vanishing suggesting that controlling for heart rate still leaves a gender difference (although some complications arise with interactions).
Interpretation of the heart rate effect is a bit tricky. One possibility is that an elevated heart rate results in a raised body temperature. Alternatively, possibly higher temperature elevates the heart rate as some sort of physiological response.